{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evolution of cooperation\n",
    "\n",
    "Code examples from [Think Complexity, 2nd edition](http://greenteapress.com/wp/complexity2), Chapter 12\n",
    "\n",
    "Copyright 2016 Allen Downey, [MIT License](http://opensource.org/licenses/MIT)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function, division\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "from copy import copy\n",
    "\n",
    "import numpy as np\n",
    "from numpy.random import rand, randint, permutation\n",
    "\n",
    "from thinkstats2 import RandomSeed\n",
    "\n",
    "import thinkplot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Previous code\n",
    "\n",
    "From the Chapter 11 notebook, we will reuse `Simulation` and `Instrument`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Simulation:\n",
    "    \n",
    "    def __init__(self, fit_land, agents):\n",
    "        \"\"\"Create the simulation:\n",
    "        \n",
    "        fit_land: fit_land\n",
    "        num_agents: int number of agents\n",
    "        agent_maker: function that makes agents\n",
    "        \"\"\"\n",
    "        self.fit_land = fit_land\n",
    "        self.agents = np.asarray(agents)\n",
    "        self.instruments = []\n",
    "        \n",
    "    def add_instrument(self, instrument):\n",
    "        \"\"\"Adds an instrument to the list.\n",
    "        \n",
    "        instrument: Instrument object\n",
    "        \"\"\"\n",
    "        self.instruments.append(instrument)\n",
    "        \n",
    "    def plot(self, index, *args, **kwargs):\n",
    "        \"\"\"Plot the results from the indicated instrument.\n",
    "        \"\"\"\n",
    "        self.instruments[index].plot(*args, **kwargs)\n",
    "        \n",
    "    def run(self, num_steps=500):\n",
    "        \"\"\"Run the given number of steps.\n",
    "        \n",
    "        num_steps: integer\n",
    "        \"\"\"\n",
    "        # initialize any instruments before starting\n",
    "        self.update_instruments()\n",
    "        \n",
    "        for _ in range(num_steps):\n",
    "            self.step()\n",
    "        \n",
    "    def step(self):\n",
    "        \"\"\"Simulate a time step and update the instruments.\n",
    "        \"\"\"\n",
    "        n = len(self.agents)\n",
    "        fits = self.get_fitnesses()\n",
    "        \n",
    "        # see who dies\n",
    "        index_dead = self.choose_dead(fits)\n",
    "        num_dead = len(index_dead)\n",
    "        \n",
    "        # replace the dead with copies of the living\n",
    "        replacements = self.choose_replacements(num_dead, fits)\n",
    "        self.agents[index_dead] = replacements\n",
    "\n",
    "        # update any instruments\n",
    "        self.update_instruments()\n",
    "        \n",
    "    def update_instruments(self):\n",
    "        for instrument in self.instruments:\n",
    "            instrument.update(self)\n",
    "            \n",
    "    def get_locs(self):\n",
    "        \"\"\"Returns a list of agent locations.\"\"\"\n",
    "        return [tuple(agent.loc) for agent in self.agents]\n",
    "    \n",
    "    def get_fitnesses(self):\n",
    "        \"\"\"Returns an array of agent fitnesses.\"\"\"\n",
    "        fits = [agent.fitness for agent in self.agents]\n",
    "        return np.array(fits)\n",
    "    \n",
    "    def choose_dead(self, ps):\n",
    "        \"\"\"Choose which agents die in the next timestep.\n",
    "        \n",
    "        ps: probability of survival for each agent\n",
    "        \n",
    "        returns: indices of the chosen ones\n",
    "        \"\"\"\n",
    "        n = len(self.agents)\n",
    "        is_dead = np.random.random(n) < 0.1\n",
    "        index_dead = np.nonzero(is_dead)[0]\n",
    "        return index_dead\n",
    "        \n",
    "    def choose_replacements(self, n, weights):\n",
    "        \"\"\"Choose which agents reproduce in the next timestep.\n",
    "        \n",
    "        n: number of choices\n",
    "        weights: array of weights\n",
    "        \n",
    "        returns: sequence of Agent objects\n",
    "        \"\"\"\n",
    "        agents = np.random.choice(self.agents, size=n, replace=True)\n",
    "        replacements = [agent.copy() for agent in agents]\n",
    "        return replacements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Instrument:\n",
    "    \"\"\"Computes a metric at each timestep.\"\"\"\n",
    "    \n",
    "    def __init__(self):\n",
    "        self.metrics = []\n",
    "        \n",
    "    def update(self, sim):\n",
    "        \"\"\"Compute the current metric.\n",
    "        \n",
    "        Appends to self.metrics.\n",
    "        \n",
    "        sim: Simulation object\n",
    "        \"\"\"\n",
    "        # child classes should implement this method\n",
    "        pass\n",
    "        \n",
    "    def plot(self, **options):\n",
    "        thinkplot.plot(self.metrics, **options)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class MeanFitness(Instrument):\n",
    "    \"\"\"Computes mean fitness at each timestep.\"\"\"\n",
    "    label = 'Mean Fitness'\n",
    "    \n",
    "    def update(self, sim):\n",
    "        mean = np.nanmean(sim.get_fitnesses())\n",
    "        self.metrics.append(mean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PD Agent\n",
    "\n",
    "The genome of an Prisoner's Dilemma-playing agent is a map from the previous choice of the opponent to the agent's next choice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Agent:\n",
    "    \n",
    "    keys = [(None, None),\n",
    "            (None, 'C'),\n",
    "            (None, 'D'),\n",
    "            ('C', 'C'),\n",
    "            ('C', 'D'),\n",
    "            ('D', 'C'),\n",
    "            ('D', 'D')]\n",
    "    \n",
    "    def __init__(self, values, fitness=np.nan):\n",
    "        \"\"\"Initialize the agent.\n",
    "        \n",
    "        values: sequence of 'C' and 'D'\n",
    "        \"\"\"\n",
    "        self.values = values\n",
    "        self.responses = dict(zip(self.keys, values))\n",
    "        self.fitness = fitness\n",
    "        \n",
    "    def reset(self):\n",
    "        \"\"\"Reset variables before a sequence of games.\n",
    "        \"\"\"\n",
    "        self.hist = [None, None]\n",
    "        self.score = 0\n",
    "        \n",
    "    def past_responses(self, num=2):\n",
    "        \"\"\"Select the given number of most recent responses.\n",
    "        \n",
    "        num: integer number of responses\n",
    "        \n",
    "        returns: sequence of 'C' and 'D'\n",
    "        \"\"\"\n",
    "        return tuple(self.hist[-num:])\n",
    "    \n",
    "    def respond(self, other):\n",
    "        \"\"\"Choose a response based on the opponent's recent responses.\n",
    "        \n",
    "        other: Agent\n",
    "        \n",
    "        returns: 'C' or 'D'\n",
    "        \"\"\"\n",
    "        key = other.past_responses()\n",
    "        resp = self.responses[key]\n",
    "        return resp\n",
    "        \n",
    "    def append(self, resp, pay):\n",
    "        \"\"\"Update based on the last response and payoff.\n",
    "        \n",
    "        resp: 'C' or 'D'\n",
    "        pay: number\n",
    "        \"\"\"\n",
    "        self.hist.append(resp)\n",
    "        self.score += pay\n",
    "        \n",
    "    prob_mutate = 0.05\n",
    "    \n",
    "    def copy(self):\n",
    "        \"\"\"Make a copy of this agent.\n",
    "        \"\"\"\n",
    "        if np.random.random() > self.prob_mutate:\n",
    "            values = self.values\n",
    "        else:\n",
    "            values = self.mutate()\n",
    "        return Agent(values, self.fitness)\n",
    "    \n",
    "    def mutate(self):\n",
    "        \"\"\"Makes a copy of this agent's values, with one mutation.\n",
    "        \n",
    "        returns: sequence of 'C' and 'D'\n",
    "        \"\"\"\n",
    "        values = list(self.values)\n",
    "        index = np.random.choice(len(values))\n",
    "        values[index] = 'C' if values[index] == 'D' else 'D'\n",
    "        return values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the genome for \"always cooperate\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_c = Agent('CCCCCCC')\n",
    "all_c.responses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And for \"always defect\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_d = Agent('DDDDDDD')\n",
    "all_d.responses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And for \"tit for tat\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tft = Agent('CCDCDCD')\n",
    "tft.responses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `copy` method has some probability of generating a mutation (in this example, `values` is initially a string; after mutation, it's a NumPy array of letters)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "RandomSeed(16)\n",
    "for i in range(10):\n",
    "    print(all_d.copy().values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.sum([all_d.copy().values != all_d.values for i in range(1000)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Tournament\n",
    "\n",
    "`Tournament` encapsulates the rules for the tournament."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Tournament:\n",
    "    \n",
    "    payoffs = {('C', 'C'): (3, 3),\n",
    "               ('C', 'D'): (0, 5),\n",
    "               ('D', 'C'): (5, 0),\n",
    "               ('D', 'D'): (1, 1)}\n",
    "    \n",
    "    num_rounds = 6\n",
    "\n",
    "    def play(self, agent1, agent2):\n",
    "        \"\"\"Play a sequence of iterated PD rounds.\n",
    "        \n",
    "        agent1: Agent\n",
    "        agent2: Agent\n",
    "\n",
    "        returns: tuple of agent1's score, agent2's score \n",
    "        \"\"\"\n",
    "        agent1.reset()\n",
    "        agent2.reset()\n",
    "        \n",
    "        for i in range(self.num_rounds):\n",
    "            resp1 = agent1.respond(agent2)\n",
    "            resp2 = agent2.respond(agent1)\n",
    "\n",
    "            pay1, pay2 = self.payoffs[resp1, resp2]\n",
    "            \n",
    "            agent1.append(resp1, pay1)\n",
    "            agent2.append(resp2, pay2)\n",
    "            \n",
    "        return agent1.score, agent2.score\n",
    "                \n",
    "    def melee(self, agents, randomize=True):\n",
    "        \"\"\"Play each agent against two others.\n",
    "        \n",
    "        Assigns the average score from the two games to agent.fitness\n",
    "        \n",
    "        agents: sequence of Agents\n",
    "        randomize: boolean, whether to shuffle the agents\n",
    "        \"\"\"\n",
    "        if randomize:\n",
    "            agents = np.random.permutation(agents)\n",
    "            \n",
    "        n = len(agents)\n",
    "        i_row = np.arange(n)\n",
    "        j_row = (i_row + 1) % n\n",
    "        \n",
    "        totals = np.zeros(n)\n",
    "        \n",
    "        for i, j in zip(i_row, j_row):\n",
    "            agent1, agent2 = agents[i], agents[j]\n",
    "            score1, score2 = self.play(agent1, agent2)\n",
    "            totals[i] += score1\n",
    "            totals[j] += score2\n",
    "            \n",
    "        for i in i_row:\n",
    "            agents[i].fitness = totals[i] / self.num_rounds / 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can test `Tournament` with a few known scenarios."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tour = Tournament()\n",
    "tour.play(all_d, all_c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tour.play(all_d, tft)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tour.play(tft, all_c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then test `melee` with a list of three agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "agents = [all_c, all_d, tft]\n",
    "agents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tour.melee(agents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this population, \"always defect\" does best."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for agent in agents:\n",
    "    print(agent.values, agent.fitness)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Probability of survival\n",
    "\n",
    "We need a function to map from points per round (0 to 5) to probability of survival (0 to 1).  I'll use a logistic curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def logistic(x, A=0, B=1, C=1, M=0, K=1, Q=1, nu=1):\n",
    "    \"\"\"Computes the generalize logistic function.\n",
    "    \n",
    "    A: controls the lower bound\n",
    "    B: controls the steepness of the transition \n",
    "    C: not all that useful, AFAIK\n",
    "    M: controls the location of the transition\n",
    "    K: controls the upper bound\n",
    "    Q: shift the transition left or right\n",
    "    nu: affects the symmetry of the transition\n",
    "    \n",
    "    returns: float or array\n",
    "    \"\"\"\n",
    "    exponent = -B * (x - M)\n",
    "    denom = C + Q * np.exp(exponent)\n",
    "    return A + (K-A) / denom ** (1/nu)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def prob_survive(scores):\n",
    "    \"\"\"Probability of survival, based on fitness.\n",
    "    \n",
    "    scores: sequence of scores, 0-60\n",
    "    \n",
    "    returns: probability\n",
    "    \"\"\"\n",
    "    return logistic(scores, A=0.7, B=1.5, M=2.5, K=0.9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scores = np.linspace(0, 5)\n",
    "probs = prob_survive(scores)\n",
    "thinkplot.plot(scores, probs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The simulator\n",
    "\n",
    "The biggest change in the simulator is in `step`, which runs `melee` to determine the fitness of each agent, and `prob_survive` to map from fitness to probability of surviving."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class PDSimulation(Simulation):\n",
    "    \n",
    "    def __init__(self, tournament, agents):\n",
    "        \"\"\"Create the simulation:\n",
    "        \n",
    "        tournament: Tournament object\n",
    "        agents: sequence of agents\n",
    "        \"\"\"\n",
    "        self.tournament = tournament\n",
    "        self.agents = np.asarray(agents)\n",
    "        self.instruments = []\n",
    "        \n",
    "    def step(self):\n",
    "        \"\"\"Simulate a time step and update the instruments.\n",
    "        \"\"\"\n",
    "        self.tournament.melee(self.agents)\n",
    "        Simulation.step(self)\n",
    "        \n",
    "    def choose_dead(self, fits):\n",
    "        \"\"\"Choose which agents die in the next timestep.\n",
    "        \n",
    "        fits: fitness of each agent\n",
    "        \n",
    "        returns: indices of the chosen ones\n",
    "        \"\"\"\n",
    "        ps = prob_survive(fits)\n",
    "        n = len(self.agents)\n",
    "        is_dead = np.random.random(n) < ps\n",
    "        index_dead = np.nonzero(is_dead)[0]\n",
    "        return index_dead"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We might be interests in starting with random agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_random_agents(n):\n",
    "    \"\"\"Make agents with random genotype.\n",
    "    \n",
    "    n: number of agents\n",
    "    \n",
    "    returns: sequence of agents\n",
    "    \"\"\"\n",
    "    agents = [Agent(np.random.choice(['C', 'D'], size=7)) \n",
    "              for _ in range(n)]\n",
    "    return agents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or with all identical agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_identical_agents(n, values):\n",
    "    \"\"\"Make agents with the given genotype.\n",
    "    \n",
    "    n: number of agents\n",
    "    values: sequence of 'C' and 'D'\n",
    "    \n",
    "    returns: sequence of agents\n",
    "    \"\"\"\n",
    "    agents = [Agent(values) for _ in range(n)]\n",
    "    return agents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are the instruments to compute various metrics.\n",
    "\n",
    "`Niceness` is the average number of `C` across the genotypes in the population."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Niceness(Instrument):\n",
    "    \"\"\"Fraction of cooperation in all genotypes.\"\"\"\n",
    "    label = 'Niceness'\n",
    "        \n",
    "    def update(self, sim):\n",
    "        responses = np.array([agent.values for agent in sim.agents])\n",
    "        metric = np.mean(responses == 'C')\n",
    "        self.metrics.append(metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Opening` is the fraction of agents that cooperate in the first round."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Opening(Instrument):\n",
    "    \"\"\"Fraction of agents that cooperate on the first round.\"\"\"\n",
    "    label = 'Opening'\n",
    "        \n",
    "    def update(self, sim):\n",
    "        responses = np.array([agent.values[0] for agent in sim.agents])\n",
    "        metric = np.mean(responses == 'C')\n",
    "        self.metrics.append(metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Retaliating` is the difference between (1) the fraction of agents that defect after the opponent defects and (2) the fraction of agents that defect after the opponent cooperates. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Retaliating(Instrument):\n",
    "    \"\"\"Tendency to defect after opponent defects.\"\"\"\n",
    "    label = 'Retaliating'\n",
    "        \n",
    "    def update(self, sim):\n",
    "        after_d = np.array([agent.values[2::2] for agent in sim.agents])\n",
    "        after_c = np.array([agent.values[1::2] for agent in sim.agents])\n",
    "        metric = np.mean(after_d == 'D') - np.mean(after_c == 'D')\n",
    "        self.metrics.append(metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Forgiving is the difference between the number of agents that cooperate after DC minus the number that cooperate after CD."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Forgiving(Instrument):\n",
    "    \"\"\"Tendency to cooperate if opponent cooperates after defecting.\"\"\"\n",
    "    label = 'Forgiving'\n",
    "        \n",
    "    def update(self, sim):\n",
    "        after_dc = np.array([agent.values[5] for agent in sim.agents])\n",
    "        after_cd = np.array([agent.values[4] for agent in sim.agents])\n",
    "        metric = np.mean(after_dc == 'C') - np.mean(after_cd == 'C')\n",
    "        self.metrics.append(metric)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Forgiving2(Instrument):\n",
    "    \"\"\"Ability to cooperate after the first two rounds.\"\"\"\n",
    "    label = 'Forgiving2'\n",
    "        \n",
    "    def update(self, sim):\n",
    "        after_two = np.array([agent.values[3:] for agent in sim.agents])\n",
    "        metric = np.mean(np.any(after_two=='C', axis=1))\n",
    "        self.metrics.append(metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results\n",
    "\n",
    "Here's a simulation that starts with 100 defectors and runs 5000 timesteps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tour = Tournament()\n",
    "\n",
    "agents = make_identical_agents(100, list('DDDDDDD'))\n",
    "sim = PDSimulation(tour, agents)\n",
    "\n",
    "sim.add_instrument(MeanFitness())\n",
    "sim.add_instrument(Niceness())\n",
    "sim.add_instrument(Opening())\n",
    "sim.add_instrument(Retaliating())\n",
    "sim.add_instrument(Forgiving())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "RandomSeed(17)\n",
    "sim.run(5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's look at some results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_result(index, **options):\n",
    "    \"\"\"Plots the results of the indicated instrument.\n",
    "    \n",
    "    index: integer\n",
    "    \"\"\"\n",
    "    sim.plot(index, **options)\n",
    "    instrument = sim.instruments[index]\n",
    "    print(np.mean(instrument.metrics[1000:]))\n",
    "    thinkplot.Config(xlabel='Time steps', \n",
    "                     ylabel=instrument.label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initially, mean fitness is 1, because that's the number of points each defector gets per round when facing another defector.\n",
    "\n",
    "After a few hundred steps, mean fitness climbs to a steady state near 2.5, although it oscillates around this point substantially.\n",
    "\n",
    "In a world of all cooperators, mean fitness would be 3, so this steady state is hardly utopia, but it's not that far off."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "colors = ['#f7fcf0','#e0f3db','#ccebc5','#a8ddb5','#7bccc4','#4eb3d3','#2b8cbe','#0868ac','#084081']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_result(0, color=colors[7])\n",
    "thinkplot.save('chap12-1', clf=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To average number of C's, across all agents and all locations in the genome, is generally more than half, with a long range mean above 0.6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_result(1, color=colors[6])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results are similar for the opening move: the fraction of agents who start out cooperating is generally more than half, with a long-range mean above 0.6.  This fraction varies widely in time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_result(2, color=colors[5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "thinkplot.preplot(cols=2)\n",
    "plot_result(1, color=colors[6])\n",
    "thinkplot.subplot(2)\n",
    "plot_result(2, color=colors[5])\n",
    "\n",
    "thinkplot.save('chap12-2', clf=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There might be a weak inclination toward retaliation, with slightly more defections after the opponent defects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_result(3, color=colors[4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of the strategies are forgiving in the sense that they have a short memory, so most of them are capable of cooperating at some point after an opponent has defected.  But there is no evidence that they are specifically more likely to forgive a defection from two rounds ago, compared to a defection in the previous round."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_result(4, color=colors[3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following cells explore the composition of the final population.  But because the distribution of agents varies so much over time, the details of a single timestep might not mean much.\n",
    "\n",
    "Here are the final genomes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for agent in sim.agents:\n",
    "    print(agent.values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here are the most common genomes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pandas import Series\n",
    "\n",
    "responses = [''.join(agent.values) for agent in sim.agents]\n",
    "Series(responses).value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise:** The simulation in this notebook depends on a number of conditions and parameters I chose arbitrarily.  As an exercise, I encourage you to explore other conditions to see what effect they have on the results.  Here are some suggestions:\n",
    "\n",
    "1.  Vary the initial conditions: instead of starting with all defectors, see what happens if you start with all cooperators, all TFT, or random agents.\n",
    "\n",
    "2.  In `Tournament.melee`, I shuffle the agents at the beginning of each time step, so each agent plays against two randomly-chosen agents.  What happens if you don't shuffle?  In that case, each agent would play against the same neighbors repeatedly.  That might make it easier for a minority strategy to invade a majority, by taking advantage of locality.\n",
    "\n",
    "3.  Since each agent only plays against two other agents, the outcome of each round is highly variable: an agent that would do well against most other agents might get unlucky during any given round, or the other way around.  What happens if you increase the number of opponents each agent plays against during each round?  Or what if an agent's fitness at the end of each step is the average of its current score and its fitness at the end of the previous round?\n",
    "\n",
    "4.  The function I chose for `prob_survival` varies from 0.7 to 0.9, so the least fit agent, with `p=0.7`, lives for `3.33` timesteps, on average, and the most fit agent lives for `10` timesteps.  What happens if you make `prob_survival` more or less \"aggressive\".\n",
    "\n",
    "5.  I chose `num_rounds=6` so that each element of the genome has roughly the same impact on the outcome of a match.  But that is substantially shorter than what Alexrod used in his tournaments.  What happens if you increase `num_rounds`?  Note: if you explore the effect of this parameter, you might also want to create an instrument to measure the niceness of the last 4 elements of the genome, which will be under more selective pressure as `num_rounds` increases.\n",
    "\n",
    "6.  My implementation has differential survival but just random reproduction.  What happens if you add differential reproduction?\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Exercise:**  In these simulations, the population never converges to a state where a majority share the same, presumably optimal, genotype.  There are two possible explanations for this outcome: one is that there is no optimal strategy, because whenever the population is dominated by a majority genotype, that condition creates an opportunity for a minority to invade; the other possibility is that the mutation rate is high enough to maintain a diversity of genotypes even if the majority is non-optimal.  To distinguish between these explanations, try lowering the mutation rate to see what happens.  Alternatively, start with a random population and run without mutation until only one genotype survives.  Or run with mutation until the system reaches something like a steady state; then turn off mutation and run until there is only one surviving genotype.  What are the characteristics of the genotypes that prevail in these conditions? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
